28 research outputs found
Spatio-temporal video autoencoder with differentiable memory
We describe a new spatio-temporal video autoencoder, based on a classic
spatial image autoencoder and a novel nested temporal autoencoder. The temporal
encoder is represented by a differentiable visual memory composed of
convolutional long short-term memory (LSTM) cells that integrate changes over
time. Here we target motion changes and use as temporal decoder a robust
optical flow prediction module together with an image sampler serving as
built-in feedback loop. The architecture is end-to-end differentiable. At each
time step, the system receives as input a video frame, predicts the optical
flow based on the current observation and the LSTM memory state as a dense
transformation map, and applies it to the current frame to generate the next
frame. By minimising the reconstruction error between the predicted next frame
and the corresponding ground truth next frame, we train the whole system to
extract features useful for motion estimation without any supervision effort.
We present one direct application of the proposed framework in
weakly-supervised semantic segmentation of videos through label propagation
using optical flow
HDRFusion:HDR SLAM using a low-cost auto-exposure RGB-D sensor
We describe a new method for comparing frame appearance in a frame-to-model
3-D mapping and tracking system using an low dynamic range (LDR) RGB-D camera
which is robust to brightness changes caused by auto exposure. It is based on a
normalised radiance measure which is invariant to exposure changes and not only
robustifies the tracking under changing lighting conditions, but also enables
the following exposure compensation perform accurately to allow online building
of high dynamic range (HDR) maps. The latter facilitates the frame-to-model
tracking to minimise drift as well as better capturing light variation within
the scene. Results from experiments with synthetic and real data demonstrate
that the method provides both improved tracking and maps with far greater
dynamic range of luminosity.Comment: 14 page
Learning Latent Space Dynamics for Tactile Servoing
To achieve a dexterous robotic manipulation, we need to endow our robot with
tactile feedback capability, i.e. the ability to drive action based on tactile
sensing. In this paper, we specifically address the challenge of tactile
servoing, i.e. given the current tactile sensing and a target/goal tactile
sensing --memorized from a successful task execution in the past-- what is the
action that will bring the current tactile sensing to move closer towards the
target tactile sensing at the next time step. We develop a data-driven approach
to acquire a dynamics model for tactile servoing by learning from
demonstration. Moreover, our method represents the tactile sensing information
as to lie on a surface --or a 2D manifold-- and perform a manifold learning,
making it applicable to any tactile skin geometry. We evaluate our method on a
contact point tracking task using a robot equipped with a tactile finger. A
video demonstrating our approach can be seen in https://youtu.be/0QK0-Vx7WkIComment: Accepted to be published at the International Conference on Robotics
and Automation (ICRA) 2019. The final version for publication at ICRA 2019 is
7 pages (i.e. 6 pages of technical content (including text, figures, tables,
acknowledgement, etc.) and 1 page of the Bibliography/References), while this
arXiv version is 8 pages (added Appendix and some extra details